Small-Variance Asymptotics for Dirichlet Process Mixtures of SVMs
نویسندگان
چکیده
Infinite SVM (iSVM) is a Dirichlet process (DP) mixture of large-margin classifiers. Though flexible in learning nonlinear classifiers and discovering latent clustering structures, iSVM has a difficult inference task and existing methods could hinder its applicability to large-scale problems. This paper presents a smallvariance asymptotic analysis to derive a simple and efficient algorithm, which monotonically optimizes a maxmargin DP-means (MDPM) problem, an extension of DP-means for both predictive learning and descriptive clustering. Our analysis is built on Gibbs infinite SVMs, an alternative DP mixture of large-margin machines, which admits a partially collapsed Gibbs sampler without truncation by exploring data augmentation techniques. Experimental results show that MDPM runs much faster than similar algorithms without sacrificing prediction accuracies.
منابع مشابه
Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models
Sampling and variational inference techniques are two standard methods for inference in probabilistic models, but for many problems, neither approach scales effectively to large-scale data. An alternative is to relax the probabilistic model into a non-probabilistic formulation which has a scalable associated algorithm. This can often be fulfilled by performing small-variance asymptotics, i.e., ...
متن کاملSmall Variance Asymptotics for Non-Parametric Online Robot Learning
Small variance asymptotics is emerging as a useful technique for inference in large scale Bayesian non-parametric mixture models. This paper analyses the online learning of robot manipulation tasks with Bayesian non-parametric mixture models under small variance asymptotics. The analysis yields a scalable online sequence clustering (SOSC) algorithm that is non-parametric in the number of cluste...
متن کاملOnline Inference in Bayesian Non-Parametric Mixture Models under Small Variance Asymptotics
Adapting statistical learning models online with large scale streaming data is a challenging problem. Bayesian non-parametric mixture models provide flexibility in model selection, however, their widespread use is limited by the computational overhead of existing sampling-based and variational techniques for inference. This paper analyses the online inference problem in Bayesian non-parametricm...
متن کاملCombinatorial Topic Models using Small-Variance Asymptotics
Modern topic models typically have a probabilistic formulation, and derive their inference algorithms based on Latent Dirichlet Allocation (LDA) and its variants. In contrast, we approach topic modeling via combinatorial optimization, and take a small-variance limit of LDA to derive a new objective function. We minimize this objective by using ideas from combinatorial optimization, obtaining a ...
متن کاملMAP for Exponential Family Dirichlet Process Mixture Models
The Dirichlet process mixture (DPM) is a ubiquitous, flexible Bayesian nonparametric model. However, full probabilistic inference in this model is analytically intractable, so that computationally intensive techniques such as Gibb’s sampling are required. As a result, DPM-based methods, which have considerable potential, are restricted to applications in which computational resources and time f...
متن کامل